ReCANVo: A database of real-world communicative and affective nonverbal vocalizations.

Nonverbal vocalizations, such as sighs, grunts, and yells, are informative expressions within typical verbal speech. Likewise, individuals who produce 0-10 spoken words or word approximations ("minimally speaking" individuals) convey rich affective and communicative information through nonverbal vocalizations even without verbal speech. Yet, despite their rich content, little to no data exists on the vocal expressions of this population. Here, we present ReCANVo: Real-World Communicative and Affective Nonverbal Vocalizations - a novel dataset of non-speech vocalizations labeled by function from minimally speaking individuals. The ReCANVo database contains over 7000 vocalizations spanning communicative and affective functions from eight minimally speaking individuals, along with communication profiles for each participant. Vocalizations were recorded in real-world settings and labeled in real-time by a close family member who knew the communicator well and had access to contextual information while labeling. ReCANVo is a novel database of nonverbal vocalizations from minimally speaking individuals, the largest available dataset of nonverbal vocalizations, and one of the only affective speech datasets collected amidst daily life across contexts.

Interventions to Modify Psychological Well-Being: Progress, Promises, and an Agenda for Future Research.

Psychological well-being, characterized by feelings, cognitions, and strategies that are associated with positive functioning (including hedonic and eudaimonic well-being), has been linked with better physical health and greater longevity. Importantly, psychological well-being can be strengthened with interventions, providing a strategy for improving population health. But are the effects of well-being interventions meaningful, durable, and scalable enough to improve health at a population-level? To assess this possibility, a cross-disciplinary group of scholars convened to review current knowledge and develop a research agenda. Here we summarize and build on the key insights from this convening, which were: (1) existing interventions should continue to be adapted to achieve a large-enough effect to result in downstream improvements in psychological functioning and health, (2) research should determine the durability of interventions needed to drive population-level and lasting changes, (3) a shift from individual-level care and treatment to a public-health model of population-level prevention is needed and will require new infrastructure that can deliver interventions at scale, (4) interventions should be accessible and effective in racially, ethnically, and geographically diverse samples. A discussion examining the key future research questions follows.

Real-time digital monitoring of a suicide attempt by a hospital patient.

Suicide is among the most devastating problems facing clinicians, who currently have limited tools to predict and prevent suicidal behavior. Here we report on real-time, continuous smartphone and sensor data collected before, during, and after a suicide attempt made by a patient during a psychiatric inpatient hospitalization. We observed elevated and persistent sympathetic nervous system arousal and suicidal thinking leading up to the suicide attempt. This case provides the highest resolution data to date on the psychological, psychophysiological, and behavioral markers of imminent suicidal behavior and highlights new directions for prediction and prevention efforts.

Rising early warning signals in affect associated with future changes in depression: a dynamical systems approach.

BACKGROUND: Predicting future states of psychopathology such as depressive episodes has been a hallmark initiative in mental health research. Dynamical systems theory has proposed that rises in certain 'early warning signals' (EWSs) in time-series data (e.g. auto-correlation, temporal variance, network connectivity) may precede impending changes in disorder severity. The current study investigates whether rises in these EWSs over time are associated with future changes in disorder severity among a group of patients with major depressive disorder (MDD). METHODS: Thirty-one patients with MDD completed the study, which consisted of daily smartphone-delivered surveys over 8 weeks. Daily positive and negative affect were collected for the time-series analyses. A rolling window approach was used to determine whether rises in auto-correlation of total affect, temporal standard deviation of total affect, and overall network connectivity in individual affect items were predictive of increases in depression symptoms. RESULTS: Results suggested that rises in auto-correlation were significantly associated with worsening in depression symptoms (r = 0.41, p = 0.02). Results indicated that neither rises in temporal standard deviation (r = -0.23, p = 0.23) nor in network connectivity (r = -0.12, p = 0.59) were associated with changes in depression symptoms. CONCLUSIONS: This study more rigorously examines whether rises in EWSs were associated with future depression symptoms in a larger group of patients with MDD. Results indicated that rises in auto-correlation were the only EWS that was associated with worsening future changes in depression.

Consensus Statement on Ethical & Safety Practices for Conducting Digital Monitoring Studies with People at Risk of Suicide and Related Behaviors.

OBJECTIVE: Digital monitoring technologies (e.g., smart-phones and wearable devices) provide unprecedented opportunities to study potentially harmful behaviors such as suicide, violence, and alcohol/substance use in real-time. The use of these new technologies has the potential to significantly advance the understanding, prediction, and prevention of these behaviors. However, such technologies also introduce myriad ethical and safety concerns, such as deciding when and how to intervene if a participant's responses indicate elevated risk during the study? METHODS: We used a modified Delphi process to develop a consensus among a diverse panel of experts on the ethical and safety practices for conducting digital monitoring studies with those at risk for suicide and related behaviors. Twenty-four experts including scientists, clinicians, ethicists, legal experts, and those with lived experience provided input into an iterative, multi-stage survey, and discussion process. RESULTS: Consensus was reached on multiple aspects of such studies, including: inclusion criteria, informed consent elements, technical and safety procedures, data review practices during the study, responding to various levels of participant risk in real-time, and data and safety monitoring. CONCLUSIONS: This consensus statement provides guidance for researchers, funding agencies, and institutional review boards regarding expert views on current best practices for conducting digital monitoring studies with those at risk for suicide-with relevance to the study of a range of other potentially harmful behaviors (e.g., alcohol/substance use and violence). This statement also highlights areas in which more data are needed before consensus can be reached regarding best ethical and safety practices for digital monitoring studies.

Guidelines for Conducting Ethical Artificial Intelligence Research in Neurology: A Systematic Approach for Clinicians and Researchers.

Preemptive recognition of the ethical implications of study design and algorithm choices in artificial intelligence (AI) research is an important but challenging process. AI applications have begun to transition from a promising future to clinical reality in neurology. As the clinical management of neurology is often concerned with discrete, often unpredictable, and highly consequential events linked to multimodal data streams over long timescales, forthcoming advances in AI have great potential to transform care for patients. However, critical ethical questions have been raised with implementation of the first AI applications in clinical practice. Clearly, AI will have far-reaching potential to promote, but also to endanger, ethical clinical practice. This article employs an anticipatory ethics approach to scrutinize how researchers in neurology can methodically identify ethical ramifications of design choices early in the research and development process, with a goal of preempting unintended consequences that may violate principles of ethical clinical care. First, we discuss the use of a systematic framework for researchers to identify ethical ramifications of various study design and algorithm choices. Second, using epilepsy as a paradigmatic example, anticipatory clinical scenarios that illustrate unintended ethical consequences are discussed, and failure points in each scenario evaluated. Third, we provide practical recommendations for understanding and addressing ethical ramifications early in methods development stages. Awareness of the ethical implications of study design and algorithm choices that may unintentionally enter AI is crucial to ensuring that incorporation of AI into neurology care leads to patient benefit rather than harm.

Smartwatch biomarkers and the path to clinical use.

Wearables have advanced from collecting consumer-quality fitness data to collecting continuous clinical-quality physiology that, when processed carefully, can identify medically significant events. In a recent issue of Nature Medicine, Dunn et al.1 described how vital signs from wearables predict clinical laboratory blood- and urine-based measurements better than vital signs measured in the clinic.

Sleep assessment by means of a wrist actigraphy-based algorithm: agreement with polysomnography in an ambulatory study on older adults.

The purpose of the present work is to examine, on a clinically diverse population of older adults (N = 46) sleeping at home, the performance of two actigraphy-based sleep tracking algorithms (i.e., Actigraphy-based Sleep algorithm, ACT-S1 and Sadeh's algorithm) compared to manually scored electroencephalography-based PSG (PSG-EEG). ACT-S1 allows for a fully automatic identification of sleep period time (SPT) and within the identified sleep period, the sleep-wake classification. SPT detected by ACT-S1 did not differ statistically from using PSG-EEG (bias = -9.98 min; correlation 0.89). In sleep-wake classification on 30-s epochs within the identified sleep period, the new ACT-S1 presented similar or slightly higher accuracy (83-87%), precision (86-89%) and F1 score (90-92%), significantly higher specificity (39-40%), and significantly lower, but still high, sensitivity (96-97%) compared to Sadeh's algorithm, which achieved 99% sensitivity as the only measure better than ACT-S1's. Total sleep times (TST) estimated with ACT-S1 and Sadeh's algorithm were higher, but still highly correlated to PSG-EEG's TST. Sleep quality metrics of sleep period efficiency and wake-after-sleep-onset computed by ACT-S1 were not significantly different from PSG-EEG, while the same sleep quality metrics derived by Sadeh's algorithm differed significantly from PSG-EEG. Agreement between ACT-S1 and PSG-EEG reached was highest when analyzing the subset of subjects with least disrupted sleep (N = 28). These results provide evidence of promising performance of a full-automation of the sleep tracking procedure with ACT-S1 on older adults. Future longitudinal validations across specific medical conditions are needed. The algorithm's performance may further improve with integrating multi-sensor information.

Advancing Neuroscience through Wearable Devices.

Faster, more reliable, and comfortably wearable personal devices are producing data from biosensors on an unprecedented scale. Combined with context and analytics, these signals hold great promise to advance neuroscience via real-world data. Here, we discuss wearable technology broadly and provide specific examples of activity patterns from electrodermal sensors found during sleep, stress, and seizures.

Forecasting stress, mood, and health from daytime physiology in office workers and students.

We examine the problem of forecasting tomorrow morning's three self-reported levels (on scales from 0 to 100) of stressed-calm, sad-happy, and sick-healthy based on physiological data (skin conductance, skin temperature, and acceleration) from a sensor worn on the wrist from 10am-5pm today. We train automated forecasting regression algorithms using Random Forests and compare their performance over two sets of data: "workers" consisting of 490 days of weekday data from 39 employees at a high-tech company in Japan and "students" consisting of 3,841 days of weekday data from 201 New England USA college students. Mean absolute errors on held-out test data achieved 10.8, 13.5, and 14.4 for the estimated levels of mood, stress, and health respectively of office workers, and 17.8, 20.3, and 20.4 for the mood, stress, and health respectively of students. Overall the two groups reported comparable stress and mood scores, while employees reported slightly poorer health, and engaged in significantly lower levels of physical activity as measured by accelerometers. We further examine differences in population features and how systems trained on each population performed when tested on the other.

Monitoring Changes in Depression Severity Using Wearable and Mobile Sensors.

Background: While preliminary evidence suggests that sensors may be employed to detect presence of low mood it is still unclear whether they can be leveraged for measuring depression symptom severity. This study evaluates the feasibility and performance of assessing depressive symptom severity by using behavioral and physiological features obtained from wristband and smartphone sensors. Method: Participants were thirty-one individuals with Major Depressive Disorder (MDD). The protocol included 8 weeks of behavioral and physiological monitoring through smartphone and wristband sensors and six in-person clinical interviews during which depression was assessed with the 17-item Hamilton Depression Rating Scale (HDRS-17). Results: Participants wore the right and left wrist sensors 92 and 94% of the time respectively. Three machine-learning models estimating depressive symptom severity were developed-one combining features from smartphone and wearable sensors, one including only features from the smartphones, and one including features from wrist sensors-and evaluated in two different scenarios. Correlations between the models' estimate of HDRS scores and clinician-rated HDRS ranged from moderate to high (0.46 [CI: 0.42, 0.74] to 0.7 [CI: 0.66, 0.74]) and had moderate accuracy with Mean Absolute Error ranging between 3.88 ± 0.18 and 4.74 ± 1.24. The time-split scenario of the model including only features from the smartphones performed the best. The ten most predictive features in the model combining physiological and mobile features were related to mobile phone engagement, activity level, skin conductance, and heart rate variability. Conclusion: Monitoring of MDD patients through smartphones and wrist sensors following a clinician-rated HDRS assessment is feasible and may provide an estimate of changes in depressive symptom severity. Future studies should further examine the best features to estimate depressive symptoms and strategies to further enhance accuracy.

Irregular sleep and event schedules are associated with poorer self-reported well-being in US college students.

STUDY OBJECTIVES: Sleep regularity, in addition to duration and timing, is predictive of daily variations in well-being. One possible contributor to changes in these sleep dimensions are early morning scheduled events. We applied a composite metric-the Composite Phase Deviation (CPD)-to assess mistiming and irregularity of both sleep and event schedules to examine their relationship with self-reported well-being in US college students. METHODS: Daily well-being, actigraphy, and timing of sleep and first scheduled events (academic/exercise/other) were collected for approximately 30 days from 223 US college students (37% females) between 2013 and 2016. Participants rated well-being daily upon awakening on five scales: Sleepy-Alert, Sad-Happy, Sluggish-Energetic, Sick-Healthy, and Stressed-Calm. A longitudinal growth model with time-varying covariates was used to assess relationships between sleep variables (i.e. CPDSleep, sleep duration, and midsleep time) and daily and average well-being. Cluster analysis was used to examine relationships between CPD for sleep vs. event schedules. RESULTS: CPD for sleep was a significant predictor of average well-being (e.g. Stressed-Calm: b = -6.3, p < 0.01), whereas sleep duration was a significant predictor of daily well-being (Stressed-Calm, b = 1.0, p < 0.001). Although cluster analysis revealed no systematic relationship between CPD for sleep vs. event schedules (i.e. more mistimed/irregular events were not associated with more mistimed/irregular sleep), they interacted upon well-being: the poorest well-being was reported by students for whom both sleep and event schedules were mistimed and irregular. CONCLUSIONS: Sleep regularity and duration may be risk factors for lower well-being in college students. Stabilizing sleep and/or event schedules may help improve well-being. CLINICAL TRIAL REGISTRATION: NCT02846077.

Stress measurement using speech: Recent advancements, validation issues, and ethical and privacy considerations.

Life stress is a well-established risk factor for a variety of mental and physical health problems, including anxiety disorders, depression, chronic pain, heart disease, asthma, autoimmune diseases, and neurodegenerative disorders. The purpose of this article is to describe emerging approaches for assessing stress using speech, which we do by reviewing the methodological advantages of these digital health tools, and the validation, ethical, and privacy issues raised by these technologies. As we describe, it is now possible to assess stress via the speech signal using smartphones and smart speakers that employ software programs and artificial intelligence to analyze several features of speech and speech acoustics, including pitch, jitter, energy, rate, and length and number of pauses. Because these digital devices are ubiquitous, we can now assess individuals' stress levels in real time in almost any natural environment in which people speak. These technologies thus have great potential for advancing digital health initiatives that involve continuously monitoring changes in psychosocial functioning and disease risk over time. However, speech-based indices of stress have yet to be well-validated against stress biomarkers (e.g., cortisol, cytokines) that predict disease risk. In addition, acquiring speech samples raises the possibility that conversations intended to be private could one day be made public; moreover, obtaining real-time psychosocial risk information prompts ethical questions regarding how these data should be used for medical, commercial, and personal purposes. Although assessing stress using speech thus has enormous potential, there are critical validation, privacy, and ethical issues that must be addressed.

Multimodal wrist-worn devices for seizure detection and advancing research: Focus on the Empatica wristbands.

Wearable automated seizure detection devices offer a high potential to improve seizure management, through continuous ambulatory monitoring, accurate seizure counts, and real-time alerts for prompt intervention. More importantly, these devices can be a life-saving help for people with a higher risk of sudden unexpected death in epilepsy (SUDEP), especially in case of generalized tonic-clonic seizures (GTCS). The Embrace and E4 wristbands (Empatica) are the first commercially available multimodal wristbands that were designed to sense the physiological hallmarks of ongoing GTCS: while Embrace only embeds a machine learning-based detection algorithm, both E4 and Embrace devices are equipped with motion (accelerometers, ACC) and electrodermal activity (EDA) sensors and both the devices received medical clearance (E4 from EU CE, Embrace from EU CE and US FDA). The aim of this contribution is to provide updated evidence of the effectiveness of GTCS detection and monitoring relying on the combination of ACM and EDA sensors. A machine learning algorithm able to recognize ACC and EDA signatures of GTCS-like events has been developed on E4 data, labeled using gold-standard video-EEG examined by epileptologists in clinical centers, and has undergone continuous improvement. While keeping an elevated sensitivity to GTCS (92-100%), algorithm improvements and growing data availability led to lower false alarm rate (FAR) from the initial ˜2 down to 0.2-1 false alarms per day, as showed by retrospective and prospective analyses in inpatient settings. Algorithm adjustment to better discriminate real-life physical activities from GTCS, has brought the initial FAR of ˜6 on outpatient real life settings, down to values comparable to best-case clinical settings (FAR < 0.5), with comparable sensitivity. Moreover, using multimodal sensing, it has been possible not only to detect GTCS but also to quantify seizure-induced autonomic dysfunction, based on automatic features of abnormal motion and EDA. The latter biosignal correlates with the duration of post-ictal generalized EEG suppression, a biomarker observed in 100% of monitored SUDEP cases.

Use of In-Game Rewards to Motivate Daily Self-Report Compliance: Randomized Controlled Trial.

BACKGROUND: Encouraging individuals to report daily information such as unpleasant disease symptoms, daily activities and behaviors, or aspects of their physical and emotional state is difficult but necessary for many studies and clinical trials that rely on patient-reported data as primary outcomes. Use of paper diaries is the traditional method of completing daily diaries, but digital surveys are becoming the new standard because of their increased compliance; however, they still fall short of desired compliance levels. OBJECTIVE: Mobile games using in-game rewards offer the opportunity to increase compliance above the rates of digital diaries and paper diaries. We conducted a 5-week randomized control trial to compare the completion rates of a daily diary across 3 conditions: a paper-based participant-reported outcome diary (Paper PRO), an electronic-based participant-reported outcome diary (ePRO), and a novel ePRO diary with in-game rewards (Game-Motivated ePRO). METHODS: We developed a novel mobile game that is a combination of the idle and pet collection genres to reward individuals who complete a daily diary with an in-game reward. Overall, 197 individuals aged 6 to 24 years (male: 100 and female: 97) were enrolled in a 5-week study after being randomized into 1 of the 3 methods of daily diary completion. Moreover, 157 participants (male: 84 and female: 69) completed at least one diary and were subsequently included in analysis of compliance rates. RESULTS: We observed a significant difference (F2,124=6.341; P=.002) in compliance to filling out daily diaries, with the Game-Motivated ePRO group having the highest compliance (mean completion 86.4%, SD 19.6%), followed by the ePRO group (mean completion 77.7%, SD 24.1%), and finally, the Paper PRO group (mean completion 70.6%, SD 23.4%). The Game-Motivated ePRO (P=.002) significantly improved compliance rates above the Paper PRO. In addition, the Game-Motivated ePRO resulted in higher compliance rates than the rates of ePRO alone (P=.09). Equally important, even though we observed significant differences in completion of daily diaries between groups, we did not observe any statistically significant differences in association between the responses to a daily mood question and study group, the average diary completion time (P=.52), or the System Usability Scale score (P=.88). CONCLUSIONS: The Game-Motivated ePRO system encouraged individuals to complete the daily diaries above the compliance rates of the Paper PRO and ePRO without altering the participants' responses. TRIAL REGISTRATION: ClinicalTrials.gov NCT03738254; http://clinicaltrials.gov/ct2/show/NCT03738254 (Archived by WebCite at http://www.webcitation.org/74T1p8u52).

Wearable Motion-Based Heart Rate at Rest: A Workplace Evaluation.

This paper studies the feasibility of using low-cost motion sensors to provide opportunistic heart rate assessments from ballistocardiographic signals during restful periods of daily life. Three wearable devices were used to capture peripheral motions at specific body locations (head, wrist, and trouser pocket) of 15 participants during five regular workdays each. Three methods were implemented to extract heart rate from motion data and their performance was compared to those obtained with an FDA-cleared device. With a total of 1358 h of naturalistic sensor data, our results show that providing accurate heart rate estimations from peripheral motion signals is possible during relatively "still" moments. In our real-life workplace study, the head-mounted device yielded the most frequent assessments (22.98% of the time under 5 beats per minute of error) followed by the smartphone in the pocket (5.02%) and the wrist-worn device (3.48%). Most importantly, accurate assessments were automatically detected by using a custom threshold based on the device jerk. Due to the pervasiveness and low cost of wearable motion sensors, this paper demonstrates the feasibility of providing opportunistic large-scale low-cost samples of resting heart rate.